Arranging passenger pickups for autonomous vehicles

ABSTRACT

Aspects of the disclosure relate to arranging a pickup between a driverless vehicle and a passenger. For instance, dispatch instructions dispatching the vehicle to a predetermined pickup area in order to pick up the passenger are received by the vehicle which begins maneuvering to the predetermined pickup area. While doing so, the vehicle receives from the passenger&#39;s client computing device the device&#39;s location. An indication that the passenger is interested in a fly-by pickup is identified. The fly-by pickup allows the passenger to safely enter the vehicle at a location outside of the predetermined pickup area and prior to the one or more processors have maneuvered the vehicle to the predetermined pickup area. The vehicle determines that the fly-by pickup is appropriate based on at least the location of the client computing device and the indication, and based on the determination, maneuvers itself in order to attempt the fly-by pickup.

BACKGROUND

Autonomous vehicles, such as vehicles that do not require a humandriver, can be used to aid in the transport of passengers or items fromone location to another. Such vehicles may operate in a fully autonomousmode where passengers may provide some initial input, such as a pickupor destination location, and the vehicle maneuvers itself to thatlocation.

When a person (or user) wants to be physically transported between twolocations via a vehicle, they may use any number of taxi services. Todate, these services typically involve a human driver who is givendispatch instructions to a location to pick up the user. These services,while useful, generally fail to provide users with real time informationabout what actions a taxi is taking on the way to pick up the user.

BRIEF SUMMARY

One aspect of the disclosure provides a method for arranging a pickupbetween a driverless vehicle and a passenger. The method includesreceiving, by one or more processors, dispatch instructions dispatchingthe vehicle to a predetermined pickup area in order to pick up thepassenger; maneuvering, by the one or more processors, the vehicletowards the predetermined pickup area to pick up the passenger; whilemaneuvering, receiving, by the one or more processors from the clientcomputing device associated with the passenger, a location of the clientcomputing device; and identifying an indication that the passenger isinterested in a fly-by pickup in real time. A fly-by pickup allows thepassenger to safely enter the vehicle at a location outside of thepredetermined pickup area and prior to the one or more processors havemaneuvered the vehicle to the predetermined pickup area. The method alsoincludes determining, by the one or more processors, that the fly-bypickup is appropriate based on at least the location of the clientcomputing device and the indication, and based on the determination,further maneuvering the vehicle, by the one or more processors, in orderto attempt the fly-by pickup.

In one example, the indication includes receiving, from the clientcomputing device, information indicating that the passenger is waitingby a curb outside of the predetermined pickup area. In another example,the method also includes sending, by the one or more processors, arequest to the client computing device for confirmation that thepassenger is interested in attempting the fly-by pickup, and receiving,by the one or more processors, user input indicating that the passengeris interested in attempting the fly-by pickup, and the indicationincludes the received user input. In another example, the method alsoincludes receiving, from a perception system of the vehicle, sensor dataindicating that the passenger has performed a task, and the indicationincludes the sensor data indicating that the passenger has performed thetask. In this example, the task includes capturing an image of thepassenger's surroundings and sending the image to the vehicle, and themethod further comprises using the image to attempt the fly-by pickup.In addition or alternatively, the task includes providing a verbaldescription of the passenger, and the method further comprises using theverbal description of the passenger to attempt the fly-by pickup. Inaddition or alternatively, the task includes playing a sound through aspeaker of the client computing device, and the method further comprisesusing, by the one or more processors, the sound to maneuver the vehicletowards the passenger in order to attempt the fly-by pickup. In additionor alternatively, the task includes walking towards the vehicle, and themethod further comprises stopping, by the one or more processors, inorder to allow the passenger to enter the vehicle. In another example,the method also includes capturing, by a camera of the vehicle, an imageof the passenger, and sending, by the one or more processors, the imageof the passenger to the passenger in order to indicate that the vehiclehas identified the passenger as being nearby the vehicle. In thisexample, the image of the passenger is sent to the client computingdevice as part of the request. In another example, the method alsoincludes receiving, by the one or more processors from the clientcomputing device, information indicating that the passenger is currentlywalking, and the indication includes the received information indicatingthat the passenger is currently walking. In another example, the methodalso includes retrieving, by the one or more processors, historicalinformation for the passenger indicating that the passenger hassuccessfully achieved at least one fly-by pickup in the past, andwherein the determining is further based on the historical information.In another example, the method also includes retrieving, by the one ormore processors, historical information for the passenger indicatingthat the passenger has not successfully achieved at least one fly-bypickup in the past, and wherein the determining is further based on thehistorical information.

Another aspect of the disclosure provides a method for arranging apickup between a driverless vehicle and a passenger. The method includessending, by one or more processors of a client computing deviceassociated with the passenger, a request for the vehicle to pick up thepassenger at a pickup area; receiving, by the one or more processors,confirmation that the vehicle has been dispatched to the pickup area;receiving after the vehicle has been dispatched, by the one or moreprocessors, location information identifying a location of the vehicleas the vehicle is moving towards the pickup area; displaying, by the oneor more processors on a display of the client computing device, thelocation of the vehicle and a location of the client computing device ona map; and identifying, by the one or more processors, an indicationthat the passenger is interested in a fly-by pickup in real time. Thefly-by pickup allows the passenger to safely enter the vehicle at alocation outside of the pickup area and prior to the vehicle havingreached the predetermined pickup area. The method also includes sending,by the one or more processors, the indication to a computing deviceassociated with the vehicle; receiving, by the one or more processors,confirmation that the vehicle will attempt the fly-by pickup; anddisplaying on the display, by the one or more processors, a notificationindicating to the passenger that the vehicle will attempt the fly-bypickup based on the received confirmation.

In one example, the method also includes receiving user input indicatinga tap at a location on the map, and wherein the indication includes alocation on the map that corresponds to the user input, such that thelocation on the map is sent to the computing device associated with thevehicle. In another example, the computing device is incorporated intothe vehicle. In another example, displaying the location of the vehicleincludes displaying a first indicator on the map and displaying thelocation of the client computing device includes displaying a secondindicator on the map. In this example, the method also includesreceiving, by the one or more processors, updated location informationfor the vehicle; determining, by the one or more processors, updatedlocation information for the client computing device using a sensor ofthe client computing device; moving the first and second indicators onthe map based on the updated location information. In addition, themethod also includes when edges of the first and second indicators aremoved to contact one another, displaying a notification on the displayindicating that the passenger is able to attempt the fly-by pickup; andin response to the notification, receiving user input indicatingconfirmation that the passenger would like to attempt the fly-by pickup.The indication includes the confirmation that the passenger would liketo attempt the fly-by pickup. In addition or alternatively, the methodalso includes receiving, from a sensor of the client computing device,information indicating that the client computing device is currentlybeing moved, and wherein the indication includes the informationindicating that the client computing device is currently moving. Inaddition or alternatively, the method also includes receiving, from thecomputing device associated with the vehicle, a request for thepassenger to perform a task in order to confirm that the passenger wouldlike to attempt the fly-by pickup, wherein the indication includesconfirmation that the task has been performed by the passenger.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of an example vehicle in accordance withaspects of the disclosure.

FIG. 2 is an example representation of detailed map information inaccordance with aspects of the disclosure.

FIGS. 3A-3D are example external views of a vehicle in accordance withaspects of the disclosure.

FIGS. 4-15 are various example screen shots and client computing devicesin accordance with aspects of the disclosure.

FIG. 16 is an example flow diagram in accordance with aspects of thedisclosure.

FIG. 17 is another example flow diagram in accordance with aspects ofthe disclosure.

DETAILED DESCRIPTION

Overview

Passenger pick-up for self-driving vehicles can be challenging due tochanging environmental conditions, the absence of a human driver, anduncertainty as to how long a vehicle may have to wait (or is able towait) for the passenger. For instance, there may be other parked cars,busy traffic, construction, or other obstacles that require thevehicle's computing devices to determine whether stopping at a givenlocation is safe, permissible, or even a possibility. In addition,without a driver to make eye contact with the passenger and gesturehis/her intention to stop at a different location, the passenger can beconfused about where to meet up with the vehicle. Moreover, given thechanging environmental conditions mentioned above, the user may beuncertain about how long the vehicle may have to wait for the user toarrive and board the vehicle. This could change relatively quickly, forinstance, based on whether a vehicle is pulling over on a busy or quietstreet.

By increasing the amount of information available to a passenger andfacilitating communications between the passenger and the computers ofthe vehicle, many of the issues discussed above can be addressed. Inaddition, this may also allow the passenger and vehicle to achieve“fly-by pickups” in real time or pickups at locations other than anoriginal pickup location. A fly-by pickup is a pickup of a passengerthat is achieved prior to when the vehicle reaches a predeterminedpickup area for the vehicle to pick up the passenger. In other words,after a vehicle is dispatched to a predetermined pickup location or areain order to pick up a passenger, a new pickup location or area isidentified for the passenger and the vehicle. If a fly-by pickup issuccessful, the passenger is safely picked up at the new pickup locationor area by the vehicle. This new pickup location or area is outside ofthe predetermined pickup area. As such, a fly-by pickup may be result oflast minute change to the pickup location or area for the passenger inview of a better or alternative pickup location or area for the vehicle,the passenger, or both.

In order to achieve the fly-by pickup, information about the vehicle'sstatus may be shared with a potential passenger (or simply passenger).At the same time, information about the passenger's location may bereported to the vehicle. Because of this exchange of information, thevehicle and/or the passenger may attempt or initiate a fly-by pickup.

During a fly-by pickup, rather than pulling into a parking area andwaiting for the passenger, the vehicle may simply double park, forinstance, stopping in a lane when safe to do so and waiting only a fewseconds for the passenger to get into the car, close the door and fastenhis or her seatbelt. If this occurs, the fly-by pickup has beensuccessful. If not, the vehicle may have to pull away and continuetowards the original pickup location. In some cases, the vehicle andpassenger may need to rearrange a new pickup location after anunsuccessful fly-by pickup attempt.

Before actually attempting a fly-by pickup, in addition to the currentlocation of the passenger's client computing device, the vehicle'scomputing devices may take into consideration current conditions, suchas whether it is a busy or quiet street and how disruptive a fly-bypickup might be. Once the vehicle's computing devices have determinedthat a fly-by pickup is safe given the current and expected (historical)conditions, the vehicle may initiate a fly-by pickup.

When initiated by a vehicle, the fly-by pickup may occur automatically.In this regard, if the passenger's reported location indicates that thepassenger is already waiting at a particular spot such as by a curb andtherefore likely ready to go, the vehicle can stop at that location.Alternatively, the vehicle may ask the user if he or she is interestedin a fly-by-pick-up in real time, for instance by displaying a popupnotification with options for the user to select to accept or deny therequest to do the fly-by pickup. In some examples, the fly-by pickupnotification can be made using visual representations of therelationship between the vehicle and the location of the passenger'sclient computing device.

A fly-by pickup may also be initiated by the passenger. The user maysignal physically to the vehicle that he or she is interested in afly-by pickup by stepping into the street. Alternatively, the passengermay initiate a fly-by pickup using his or her client computing device.

To achieve a fly-by pickup, the passenger may be asked to perform sometask to further facilitate the fly-by pickup. This may include a specialgesture, which the vehicle's computing devices could detect either fromthe laser, camera, the passenger's client computing device, or somecombination of the above. In another example, the passenger may be askedto take a picture, hold his or her client computing device in aparticular way, walk to a certain location, provide verbal cues, reviewand provide feedback regarding an image captured by a camera of thevehicle, etc.

In addition to considering the current location of the passenger'sclient computing device and current and expected conditions, thevehicle's computing devices may also consider the passenger's pasthistory with attempting and achieving fly-by pickups. In some cases, thepassenger may be offered discounts, free rides, or other perks inexchange for accepting or initiating a fly-by pickup. This may also bedependent upon whether the fly-by pickup was successful (and thepassenger made it into the vehicle).

Fly-by pickups can be extremely helpful to both passengers and vehicles.For instance, a fly-by pickup can save both the vehicle and passengertime and effort to reach the original pickup location. This increasesthe efficiency of the vehicles and reduces the time needed to find aspot to park, stop and park the vehicle, and to wait for the passenger.

Example Systems

As shown in FIG. 1, a vehicle 100 in accordance with one aspect of thedisclosure includes various components. While certain aspects of thedisclosure are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, busses, recreational vehicles,etc. The vehicle may have one or more computing devices, such ascomputing device 110 containing one or more processors 120, memory 130and other components typically present in general purpose computingdevices.

The memory 130 stores information accessible by the one or moreprocessors 120, including instructions 132 and data 134 that may beexecuted or otherwise used by the processor 120. The memory 130 may beof any type capable of storing information accessible by the processor,including a computing device-readable medium, or other medium thatstores data that may be read with the aid of an electronic device, suchas a hard-drive, memory card, ROM, RAM, DVD or other optical disks, aswell as other write-capable and read-only memories. Systems and methodsmay include different combinations of the foregoing, whereby differentportions of the instructions and data are stored on different types ofmedia.

The instructions 132 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computingdevice code on the computing device-readable medium. In that regard, theterms “instructions” and “programs” may be used interchangeably herein.The instructions may be stored in object code format for directprocessing by the processor, or in any other computing device languageincluding scripts or collections of independent source code modules thatare interpreted on demand or compiled in advance. Functions, methods androutines of the instructions are explained in more detail below.

The data 134 may be retrieved, stored or modified by processor 120 inaccordance with the instructions 132. As an example, data 134 of memory130 may store predefined scenarios. A given scenario may identify a setof scenario requirements including a type of object, a range oflocations of the object relative to the vehicle, as well as otherfactors such as whether the autonomous vehicle is able to maneuveraround the object, whether the object is using a turn signal, thecondition of a traffic light relevant to the current location of theobject, whether the object is approaching a stop sign, etc. Therequirements may include discrete values, such as “right turn signal ison” or “in a right turn only lane”, or ranges of values such as “havingan heading that is oriented at an angle that is 30 to 60 degrees offsetfrom a current path of vehicle 100.” In some examples, the predeterminedscenarios may include similar information for multiple objects.

The one or more processor 120 may be any conventional processors, suchas commercially available CPUs. Alternatively, the one or moreprocessors may be a dedicated device such as an ASIC or otherhardware-based processor. Although FIG. 1 functionally illustrates theprocessor, memory, and other elements of computing device 110 as beingwithin the same block, it will be understood by those of ordinary skillin the art that the processor, computing device, or memory may actuallyinclude multiple processors, computing devices, or memories that may ormay not be stored within the same physical housing. As an example,internal electronic display 152 may be controlled by a dedicatedcomputing device having its own processor or central processing unit(CPU), memory, etc. which may interface with the computing device 110via a high-bandwidth or other network connection. In some examples, thiscomputing device may be a user interface computing device which cancommunicate with a user's client device. Similarly, the memory may be ahard drive or other storage media located in a housing different fromthat of computing device 110. Accordingly, references to a processor orcomputing device will be understood to include references to acollection of processors or computing devices or memories that may ormay not operate in parallel.

Computing device 110 may all of the components normally used inconnection with a computing device such as the processor and memorydescribed above as well as a user input 150 (e.g., a mouse, keyboard,touch screen and/or microphone) and various electronic displays (e.g., amonitor having a screen or any other electrical device that is operableto display information). In this example, the vehicle includes aninternal electronic display 152 as well as one or more speakers 154 toprovide information or audio visual experiences. In this regard,internal electronic display 152 may be located within a cabin of vehicle100 and may be used by computing device 110 to provide information topassengers within the vehicle 100. In addition to internal speakers, theone or more speakers 154 may include external speakers that are arrangedat various locations on the vehicle in order to provide audiblenotifications to objects external to the vehicle 100.

In one example, computing device 110 may be an autonomous drivingcomputing system incorporated into vehicle 100. The autonomous drivingcomputing system may capable of communicating with various components ofthe vehicle. For example, returning to FIG. 1, computing device 110 maybe in communication with various systems of vehicle 100, such asdeceleration system 160 (for controlling braking of the vehicle),acceleration system 162 (for controlling acceleration of the vehicle),steering system 164 (for controlling the orientation of the wheels anddirection of the vehicle), signaling system 166 (for controlling turnsignals), navigation system 168 (for navigating the vehicle to alocation or around objects), positioning system 170 (for determining theposition of the vehicle), perception system 172 (for detecting objectsin the vehicle's environment), and power system 174 (for example, abattery and/or gas or diesel powered engine) in order to control themovement, speed, etc. of vehicle 100 in accordance with the instructions134 of memory 130 in an autonomous driving mode which does not requireor need continuous or periodic input from a passenger of the vehicle.Again, although these systems are shown as external to computing device110, in actuality, these systems may also be incorporated into computingdevice 110, again as an autonomous driving computing system forcontrolling vehicle 100.

The computing device 110 may control the direction and speed of thevehicle by controlling various components. By way of example, computingdevice 110 may navigate the vehicle to a destination location completelyautonomously using data from the map information and navigation system168. Computer 110 may use the positioning system 170 to determine thevehicle's location and perception system 172 to detect and respond toobjects when needed to reach the location safely. In order to do so,computer 110 may cause the vehicle to accelerate (e.g., by increasingfuel or other energy provided to the engine by acceleration system 162),decelerate (e.g., by decreasing the fuel supplied to the engine,changing gears, and/or by applying brakes by deceleration system 160),change direction (e.g., by turning the front or rear wheels of vehicle100 by steering system 164), and signal such changes (e.g., by lightingturn signals of signaling system 166). Thus, the acceleration system 162and deceleration system 162 may be a part of a drivetrain that includesvarious components between an engine of the vehicle and the wheels ofthe vehicle. Again, by controlling these systems, computer 110 may alsocontrol the drivetrain of the vehicle in order to maneuver the vehicleautonomously.

As an example, computing device 110 may interact with decelerationsystem 160 and acceleration system 162 in order to control the speed ofthe vehicle. Similarly, steering system 164 may be used by computingdevice 110 in order to control the direction of vehicle 100. Forexample, if vehicle 100 configured for use on a road, such as a car ortruck, the steering system may include components to control the angleof wheels to turn the vehicle. Signaling system 166 may be used bycomputing device 110 in order to signal the vehicle's intent to otherdrivers or vehicles, for example, by lighting turn signals or brakelights when needed.

Navigation system 168 may be used by computing device 110 in order todetermine and follow a route to a location. In this regard, thenavigation system 168 and/or data 132 may store map information, e.g.,highly detailed maps that computing devices 110 can use to navigate orcontrol the vehicle. As an example, these maps may identify the shapeand elevation of roadways, lane markers, intersections, crosswalks,speed limits, traffic signal lights, buildings, signs, real time trafficinformation, vegetation, or other such objects and information. The lanemarkers may include features such as solid or broken double or singlelane lines, solid or broken lane lines, reflectors, etc. A given lanemay be associated with left and right lane lines or other lane markersthat define the boundary of the lane. Thus, most lanes may be bounded bya left edge of one lane line and a right edge of another lane line.

FIG. 2 is an example of map information 200 for a section of roadwayincluding intersections 202 and 204. In this example, the mapinformation 200 includes information identifying the shape, location,and other characteristics of lane lines 210, 212, 214, traffic signallights 220, 222, crosswalks 230, 232, sidewalks 240, stop signs 250,252, and yield sign 260. Areas where the vehicle can drive may beassociated with one or more rails 270, 272, and 274 which indicate thelocation and direction in which a vehicle should generally travel atvarious locations in the map information. For example, a vehicle mayfollow rail 270 when driving in the lane between lane lines 210 and 212,and may transition to rail 272 in order to make a right turn atintersection 204. Thereafter the vehicle may follow rail 274. Of course,given the number and nature of the rails only a few are depicted in mapinformation 200 for simplicity and ease of understanding. Although notshown, the detailed map information may also include historicalinformation identifying typical and historical traffic conditions atvarious dates and times.

Although the detailed map information is depicted herein as animage-based map, the map information need not be entirely image based(for example, raster). For example, the detailed map information mayinclude one or more roadgraphs or graph networks of information such asroads, lanes, intersections, and the connections between these features.Each feature may be stored as graph data and may be associated withinformation such as a geographic location and whether or not it islinked to other related features, for example, a stop sign may be linkedto a road and an intersection, etc. In some examples, the associateddata may include grid-based indices of a roadgraph to allow forefficient lookup of certain roadgraph features.

FIGS. 3A-3D are examples of external views of vehicle 100. As can beseen, vehicle 100 includes many features of a typical vehicle such asheadlights 302, windshield 303, taillights/turn signal lights 304, rearwindshield 305, doors 306, side view mirrors 308, tires and wheels 310,and turn signal/parking lights 312. Headlights 302, taillights/turnsignal lights 304, and turn signal/parking lights 312 may be associatedthe signaling system 166. Light bar 307 may also be associated with thesignaling system 166. As noted above, vehicle 100 may include variousspeakers 314 arranged on the external surfaces of the vehiclecorresponding to the one or more speakers 154 as noted above.

The one or more computing devices 110 of vehicle 100 may also receive ortransfer information to and from other computing devices. FIGS. 4 and 5are pictorial and functional diagrams, respectively, of an examplesystem 400 that includes a plurality of computing devices 410, 420, 430,440 and a storage system 450 connected via a network 440. System 400also includes vehicle 100, and vehicle 100A which may be configuredsimilarly to vehicle 100. Although only a few vehicles and computingdevices are depicted for simplicity, a typical system may includesignificantly more.

As shown in FIG. 4, each of computing devices 410, 420, 430, 440 mayinclude one or more processors, memory, data and instructions. Suchprocessors, memories, data and instructions may be configured similarlyto one or more processors 120, memory 130, data 132, and instructions134 of computing device 110.

The network 440, and intervening nodes, may include variousconfigurations and protocols including short range communicationprotocols such as Bluetooth, Bluetooth LE, the Internet, World Wide Web,intranets, virtual private networks, wide area networks, local networks,private networks using communication protocols proprietary to one ormore companies, Ethernet, WiFi and HTTP, and various combinations of theforegoing. Such communication may be facilitated by any device capableof transmitting data to and from other computing devices, such as modemsand wireless interfaces.

In one example, one or more computing devices 110 may include a serverhaving a plurality of computing devices, e.g., a load balanced serverfarm, that exchange information with different nodes of a network forthe purpose of receiving, processing and transmitting the data to andfrom other computing devices. For instance, one or more computingdevices 210 may include one or more server computing devices that arecapable of communicating with one or more computing devices 110 ofvehicle 100 or a similar computing device of vehicle 100A as well asclient computing devices 420, 430, 440 via the network 440. For example,vehicles 100 and 100A may be a part of a fleet of vehicles that can bedispatched by server computing devices to various locations. In thisregard, the vehicles of the fleet may periodically send the servercomputing devices location information provided by the vehicle'srespective positioning systems and the one or more server computingdevices may track the locations of the vehicles.

In addition, server computing devices 410 may use network 440 totransmit and present information to a user, such as user 422, 432, 442on a display, such as displays 424, 434, 444 of computing devices 420,430, 440. In this regard, computing devices 420, 430, 440 may beconsidered client computing devices.

As shown in FIG. 5, each client computing device 420, 430, 440 may be apersonal computing device intended for use by a user 422, 432, 442, andhave all of the components normally used in connection with a personalcomputing device including a one or more processors (e.g., a centralprocessing unit (CPU)), memory (e.g., RAM and internal hard drives)storing data and instructions, a display such as displays 424, 434, 444(e.g., a monitor having a screen, a touch-screen, a projector, atelevision, or other device that is operable to display information),and user input devices 426, 436, 446 (e.g., a mouse, keyboard,touch-screen or microphone). The client computing devices may alsoinclude a camera for recording video streams, speakers, a networkinterface device, and all of the components used for connecting theseelements to one another.

Although the client computing devices 420, 430, and 440 may eachcomprise a full-sized personal computing device, they may alternativelycomprise mobile computing devices capable of wirelessly exchanging datawith a server over a network such as the Internet. By way of exampleonly, client computing device 420 may be a mobile phone or a device suchas a wireless-enabled PDA, a tablet PC, a wearable computing device orsystem, or a netbook that is capable of obtaining information via theInternet or other networks. In another example, client computing device430 may be a wearable computing system, shown as a head-mountedcomputing system in FIG. 5. As an example the user may input informationusing a small keyboard, a keypad, microphone, using visual signals witha camera, or a touch screen.

In some examples, client computing device 440 may be concierge workstation used by an administrator to provide concierge services to userssuch as users 422 and 432. For example, a concierge 442 may use theconcierge work station 440 to communicate via a telephone call or audioconnection with users through their respective client computing devicesor vehicles 100 or 100A in order to ensure the safe operation ofvehicles 100 and 100A and the safety of the users as described infurther detail below. Although only a single concierge work station 440is shown in FIGS. 4 and 5, any number of such work stations may beincluded in a typical system.

Storage system 450 may store various types of information as describedin more detail below. This information may be retrieved or otherwiseaccessed by a server computing device, such as one or more servercomputing devices 410, in order to perform some or all of the featuresdescribed herein. For example, the information may include user accountinformation such as credentials (e.g., a user name and password as inthe case of a traditional single-factor authentication as well as othertypes of credentials typically used in multi-factor authentications suchas random identifiers, biometrics, etc.) that can be used to identify auser to the one or more server computing devices. The user accountinformation may also include personal information such as the user'sname, contact information, identifying information of the user's clientcomputing device (or devices if multiple devices are used with the sameuser account), as well as one or more unique signals for the user.

The storage system 450 may also store routing data for generating andevaluating routes between locations. For example, the routinginformation may be used to estimate how long it would take a vehicle ata first location to reach a second location. In this regard, the routinginformation may include map information, not necessarily as particularas the detailed map information described above, but including roads, aswell as information about those road such as direction (one way, twoway, etc.), orientation (North, South, etc.), speed limits, as well astraffic information identifying expected traffic conditions, etc.

The storage system 150 may also store information which can be providedto client computing devices for display to a user. For instance, thestorage system 150 may store predetermined distance information fordetermining an area at which a vehicle is likely to stop for a givenpickup or destination location. The storage system 150 may also storegraphics, icons, and other items which may be displayed to a user asdiscussed below.

As with memory 130, storage system 250 can be of any type ofcomputerized storage capable of storing information accessible by theserver computing devices 410, such as a hard-drive, memory card, ROM,RAM, DVD, CD-ROM, write-capable, and read-only memories. In addition,storage system 450 may include a distributed storage system where datais stored on a plurality of different storage devices which may bephysically located at the same or different geographic locations.Storage system 450 may be connected to the computing devices via thenetwork 440 as shown in FIG. 4 and/or may be directly connected to orincorporated into any of the computing devices 110, 410, 420, 430, 440,etc.

Example Methods

In addition to the operations described above and illustrated in thefigures, various operations will now be described. It should beunderstood that the following operations do not have to be performed inthe precise order described below. Rather, various steps can be handledin a different order or simultaneously, and steps may also be added oromitted.

In one aspect, a user may download an application for requesting avehicle to a client computing device. For example, users 122 and 132 maydownload the application via a link in an email, directly from awebsite, or an application store to client computing devices 120 and130. For example, client computing device may transmit a request for theapplication over the network, for example, to one or more servercomputing devices 110, and in response, receive the application. Theapplication may be installed locally at the client computing device.

The user may then use his or her client computing device to access theapplication and request a vehicle. As an example, a user such as user132 may use client computing device 130 to send a request to one or moreserver computing devices 110 for a vehicle. As part of this, the usermay identify a pickup location, a destination location, and, in somecases, one or more intermediate stopping locations anywhere within aservice area where a vehicle can stop.

These pickup and destination locations may be predefined (e.g., specificareas of a parking lot, etc.) or may simply be any location within aservice area of the vehicles. As an example, a pickup location can bedefaulted to current location of the user's client computing device, orcan be input by the user at the user's client device. For instance, theuser may enter an address or other location information or select alocation on a map to select a pickup location. As shown in FIG. 6A, user422 may use his or her finger 622 to tap on a map 624 displayed on thedisplay 424 of client computing device 420. In response, as shown inFIG. 6B, the location of the tap on the map, displayed as map marker626, may be identified as a requested location. Allowing the user toinput or select a location may be especially helpful where the user isnot currently located at the pickup location but will be by the time thevehicle arrives.

In the example of FIG. 6C, a user is provided with a plurality ofoptions for inputting locations. As shown, the user is able to selectfrom a series of saved locations under a saved option 630 previouslysaved by the user as discussed above. The user may also be provided withoption 640 which provide the user with the ability to view a list ofrecent locations. By selecting option 650, the user may be able toconduct a location search. For example, a user may enter a search query(“fast food restaurant” or “doctor doe” or “gas station near me”) andreceive a set of locations corresponding to the search query as with atypical map or location-based search engine.

Once the user has selected one or more of a pickup and/or destinationlocations, the client computing device 420 may send the location orlocations to one or more server computing devices of the centralizeddispatching system. In response, one or more server computing devices,such as server computing device 110, may select a vehicle, for instancebased on availability and proximity to the user. The server computingdevice may then dispatch the selected vehicle to pickup to the user byproviding the vehicle with the pickup and/or destination locationsspecified by the user.

When a vehicle, such as vehicle 101, is dispatched to the user, theclient computing device 420 can show information about where the pickuplocation will be. This information may be provided to the clientcomputing device from the server computing device 110. For example, theserver computing device 110 may access the information of storage system150 in order to identify a map and determine a route along the mapbetween the vehicle's current location and the pickup location. Thisinformation, along with a potential pickup area or zone, may then beprovided to the client computing device for display to a user.

Once received by the client computing device, the information may bedisplayed as shown in FIG. 7. For instance, client computing device 120displays a map 724 (corresponding to map 624) identifying a potentialpickup zone 702 around the pickup location identified by marker 704. Inaddition to the map and potential pickup zone 702, the client computingdevice may also display a marker 706 indicating the current location ofthe client computing device. The current location of the clientcomputing device may be determined using various methods such includingGPS and/or other location services at the passenger's client computingdevice.

As noted above, information about the vehicle's status may be sharedwith a potential passenger, or simply passenger. This may include, forinstance, displaying the vehicle's up to date location and route to thepassenger on the map so that the passenger can plan ahead how he or shewill meet the vehicle. For instance, a passenger viewing the displayedmap may be able to readily recognize whether he or she needs to crossthe street, or move to where the car is likely to find parking along theroute. As shown in FIG. 8, marker 802 identifies the current location ofthe vehicle 101. In this example, the route the vehicle is currentlytaking towards the potential pickup zone 702 is also identified by path804.

In addition, the passenger can be provided with narrative updates aboutwhat the vehicle is currently doing. For instance, as shown in FIGS. 7and 8, narrative updates 706 and 806 are displayed which indicate thestatus of the vehicle as being dispatched and on the way to pick up thepassenger. Other examples of narrative updates may include such as“Looking for a spot to pull over . . . ”, “Found a spot to pull over”,“Can't find a spot, circling around the block . . . ”, etc.

At the same time, information about the passenger's location may bereported to the vehicle. For example, the location determined using GPSand/or other location services at the passenger's client computingdevice may be sent directly (or indirectly by way of a dispatchingserver computing device) to the vehicle.

Because of this exchange of information, the vehicle and/or passengermay attempt or initiate a fly-by pickup. As noted above, during a fly-bypickup, rather than pulling into a parking area and waiting for thepassenger, the vehicle may simply double park, for instance, stopping ina lane when safe to do so and waiting only a few seconds for thepassenger to get into the car, close the door and fasten his or herseatbelt. If this occurs, the fly-by pickup has been successful. If not,the vehicle may have to pull away and continue towards the originalpickup location. In some cases, the vehicle and passenger may need torearrange a new pickup location after an unsuccessful fly-by pickupattempt.

Before actually attempting a fly-by pickup, in addition to the currentlocation of the passenger's client computing device, the vehicle'scomputing devices may take into consideration current trafficconditions, such as whether it is a busy or quiet street and howdisruptive a fly-by pickup might be. This may include processing realtime data from sensors of the vehicle as well as querying the mapinformation of the navigation system 168 for historical information todetermine how safe a street/location is (or is expected to be) forpulling over. Of course, because a fly-by pickup does not involvestopping and waiting for the passenger for long periods (more than a fewseconds), the vehicle may be able to do a fly-by pickup even on streetswhere the vehicle would normally not be able to stop or pick up apassenger.

Once the vehicle's computing devices have determined that a fly-bypickup is safe given the current and expected (historical) conditions,the vehicle may initiate a fly-by pickup. When initiated by a vehicle,the fly-by pickup may occur automatically. In this regard, if thepassenger's reported location indicates that the passenger is alreadywaiting at a particular spot such as by a curb and therefore likelyready to go, the vehicle can stop at that location. For instance, asshown in FIG. 9, the marker 706, representing the current location ofthe client computing device 420 (and presumably also the currentlocation of user 422), indicates that the passenger is very close to acurb and may be ready to be pickup up. As this information is shareddirectly with the vehicle's computing devices 110, the vehicle may usethis information to initiate a fly-by pickup. At the same time, thevehicle's computing devices may send a notification to the clientcomputing device (either directly or via server computing devices 410)indicating that the vehicle is attempting an fly-by pickup. Thenotification may be displayed on the display of the client computingdevice in order to let the passenger know that the vehicle is attemptingthe fly-by pickup.

Alternatively, rather than initiating the fly-by pickup automatically,the vehicle's computing devices may use the indication as a basis to askthe user if he or she is interested in a fly-by-pick-up in real time. Inthis regard, the vehicle's computing devices 110 may send a request tothe client computing device 420 to query whether the passenger isinterested in a fly-by pickup. At the client computing device 420, thismay be achieved as shown in FIG. 10 by displaying a popup notification1002 with options 1004 and 1006 for the user to select to accept (“YES”option 1004) or deny (“NO” option 1006) the request to do the fly-bypickup. Of course, such notifications should only be used once or twicefor any given pickup in order to avoid cluttering the screen of theclient computing device and discouraging passengers to attempt fly-bypickups. In this example, narrative update 1108 provides further detailson why the vehicle's computing devices are requesting a fly-by pickup,again to provide information about the status of the vehicle. Again,once confirmed by the user, the vehicle's computing devices may send anotification to the client computing device (either directly or viaserver computing devices 410) indicating that the vehicle is attemptingan fly-by pickup. The notification may be displayed on the display ofthe client computing device in order to let the passenger know that thevehicle is attempting the fly-by pickup.

In some examples, the fly-by pickup notification can be made usingvisual representations of the relationship between the vehicle and thelocation of the passenger's client computing device. For instance, asshown in FIG. 11, the location of the vehicle and the passenger's clientcomputing device may be represented by bubbles 1102 and 1104,respectively. As the passenger's client computing device and the vehicleapproach one another, the bubbles will move towards one another. Oncethe bubbles “meet” as shown in FIG. 12, the display may change, forinstance, by flashing once or twice and displaying the notification (asshown in FIG. 10) asking if the passenger wants to attempt a fly-bypickup. Again, once confirmed by the user, the vehicle's computingdevices may send a notification to the client computing device (eitherdirectly or via server computing devices 410) indicating that thevehicle is attempting an fly-by pickup. The notification may bedisplayed on the display of the client computing device in order to letthe passenger know that the vehicle is attempting the fly-by pickup.

In some instances, the fly-by pickup may be initiated by the passenger.For instance, as indicated above, the passenger may be able to view thecurrent location of the vehicle relative to his or her location (orrather the location of the passenger's client computing device) via amap displayed on the passenger's client computing device. This can givethe passenger a clear indication of when a fly-by pickup is possible andeven encourage him or her to initiate fly-by pickups as discussed below.

In one example, the user may also signal physically to the vehicle thathe or she is interested in a fly-by pickup by physically stepping intothe street. In this example, the vehicle's perception system 172 maydetect the downward motion of the passenger stepping into the street,and the computing devices 110 may use this information to determine thatthe passenger is interested in a fly-by pickup. At this point, thepassenger may be prompted (if the fly-by pickup is determined to be safegiven the current and expected (historical) conditions as noted above)as shown in FIG. 10 and a confirmation notification displayed to theuser indicating that the vehicle is attempting a fly-by pickup.Alternatively, rather than requesting confirmation from the user, theclient computing device may display the notification that the vehicle isattempting a fly-by pickup and the vehicle may simply stop at a locationproximate to the passenger.

Alternatively, the passenger may initiate a fly-by pickup using his orher client computing device. For instance, the passenger may tap ordouble tap on a map to pick an alternative pickup location as shown inFIGS. 13 and 14. For instance, in FIG. 13, user 422's finger 622 doubletaps on the display. In response, a marker 1402 appears indicating thelocation of one of the first or second taps. The geographic location ofthe marker relative to the map 724 is sent to the vehicle's computingdevice 110 (directly or indirectly via the server computing devices410). While this is occurring, the narrative update 1406 may bedisplayed to indicate that the client computing device is checking withthe vehicle's computing devices 110 to confirm that the fly-by pickupcan and/or will be attempted. Again, at this point, the passenger may beprompted (if the fly-by pickup is determined to be safe given thecurrent and expected (historical) conditions as noted above) as shown inFIG. 10 and a confirmation notification displayed to the user indicatingthat the vehicle is attempting a fly-by pickup. Alternatively, ratherthan requesting confirmation from the user, the client computing devicemay display the notification that the vehicle is attempting a fly-bypickup and the vehicle may simply stop at a location proximate to thepassenger.

In another example, the passenger may drag a marker representing thecurrent pickup location to another point on the map, or draw a line onthe map from his or her current location towards a street in order tosignal path that the user is going to take. As can be seen between theexamples of FIGS. 8 and 14, marker 704 of FIG. 8 has been dragged acrossthe display 422 to a new position in FIG. 14. The geographic location ofthe new position may be sent to the vehicle's computing devices 110 andat the same time, the narrative update 1406 may be displayed. Again, thepassenger may be prompted (if the fly-by pickup is determined to be safegiven the current and expected (historical) conditions as noted above)as shown in FIG. 10 and a confirmation notification displayed to theuser indicating that the vehicle is attempting a fly-by pickup.Alternatively, rather than requesting confirmation from the user, theclient computing device may display the notification that the vehicle isattempting a fly-by pickup and the vehicle may simply stop at a locationproximate to the passenger.

Once the vehicle is attempting the fly-by pickup, in order to achievethe fly-by pickup, the passenger may be asked to perform some task tofurther facilitate the fly-by pickup. This may include a specialgesture, which the vehicle's computing devices could detect either fromthe laser, camera, the passenger's client computing device, or somecombination of the above. In another example, the passenger may be askedto take a selfie so vehicle's computing devices can recognize him orher, or to take a picture or video of the area near the passenger (e.g.buildings, vegetation, etc.) so the vehicle can recognize the exactlocation of the passenger. In yet another example, the passenger may beasked to hold his or her client computing device out while the device'sscreen flashes or a flash is lit in order to allow the vehicle'scomputing devices to identify the passenger's exact location visually.Similarly, the passenger's client computing device may play a particularsound which a microphone on the vehicle captures and is used todetermine the location of the passenger. In addition or alternatively,the passenger may be asked to walk towards the vehicle, such as into theroad in front of the vehicle.

These tasks may also include asking the passenger to provide verbalcues. For instance, the passenger may speak while an external microphoneon the vehicle picks up the sound and matches the sound to a signalreceived at the microphone on the user's client computing device. Inanother example, the passenger may provide a verbal description of hisor herself so that the vehicle's computing devices can recognize thepassenger. In yet another example, if the vehicle determines that it isunable to pull over where the passenger is standing, the vehicle canopen a window and slow down while the passenger speaks instructions intothe cabin of the vehicle.

For passengers with visual impairments, the vehicle may provide audiblecues to assist the passenger and the vehicle in achieving a fly-bypickup. For instance, the vehicle could make (by playing through aspeaker) or send audio ques to the passenger's client computing devicesuch as a specific beeping sound to guide or lead the passenger towardsthe vehicle. If the vehicle's perception system is able to localize thepassenger very well, the vehicle's computing devices may provide morespecific audible instructions to the client computing device to beplayed to the user such as “walk to your right along the sidewalk for 23feet; then, step off the curb to your vehicle, which is parked 3 feetinto the road.”

In other examples, the vehicle's computing devices may send thepassenger's client computing device a picture. This picture may havebeen captured by a camera of the vehicle's perception system 172. Thepicture may be displayed on the user's client computing device with anotification indicating to the passenger that the vehicle has identifiedhim or her as being nearby. In some instances, the notification mayinclude a request for the passenger to confirm whether the vehicle hasidentified the correct person. For example, as shown in FIG. 15, image1502 of people 1504, 1508 is displayed on the display 424 with narrativeupdate 1506 indicating the vehicle has captured an image which mayinclude user 422. The narrative update also requests that the user tapon the image of his or herself to confirm that the user is actuallyincluded in the image and which of people 1504, 1508 is the user 422.The picture may be sent as part of the notification/request to achievethe fly-by pickup or as part of the typical process for picking up apassenger.

To further determine whether a passenger is ready for a fly-by pickup,the passenger's client computing device may send information to thevehicle's computing device about whether the client computing device iscurrently moving. Movement information may be provided by anaccelerometer and/or gyroscope of the client computing device. Thisalone or combined with information from the vehicle's camera mayindicate whether the passenger is currently walking or standing. In thatregard, if a passenger is standing, he or she may be more interested inbeing pickup at the original pickup location. If walking, he or she maybe more interested in a fly-by pickup. Of course, the reverse may alsobe considered true in certain circumstances.

In addition to considering the current location of the passenger'sclient computing device and current and expected conditions, thevehicle's computing devices may also consider the passenger's pasthistory with attempting and achieving fly-by pickups. For instance, ifthe passenger has little or no history of attempts, the vehicle'scomputing devices may be more willing to attempt a fly-by pickup than ifthe same passenger has a history of failing to meet the vehicle afteraccepting or requesting a fly-by pickup. Of course, if a passenger has ahistory of being successful at achieving fly-by pickups, the passengermay be more likely to receive notifications to attempt a fly-by pickup.Similarly, for a passenger who is more successful at achieving fly-bypickups, the vehicle's computing device may attempt riskier fly-bypickups than for someone who has not attempted a fly-by pickup or hasbeen unsuccessful in the past.

In some cases, the passenger may be offered discounts, free rides, orother perks in exchange for accepting or initiating a fly-by pickup.This may also be dependent upon whether the fly-by pickup was successful(and the passenger made it into the vehicle).

FIG. 16 is an example flow diagram 1600 in accordance with some of theaspects described above that may be performed by one or more computingdevices such as client computing device 120. For example, at block 1602,a request for a driverless vehicle to pick up a passenger at a pickuparea is received. Confirmation that the vehicle has been dispatched tothe pickup area is received at block 1604. After the vehicle has beendispatched, location information identifying a location of the vehicleas the vehicle is moving towards the pickup area is received at block1606. The location of the vehicle and a location of a client computingdevice of the passenger are displayed on a map on a display of theclient computing device at block 1608. An indication that the passengeris interested in a fly-by pickup in real time is identified at block1610. The fly-by pickup allows the passenger to safely enter the vehicleat a location outside of the pickup area and prior to the vehicle havingreached the predetermined pickup area. The indication is sent to acomputing device associated with the vehicle at block 1612. Confirmationthat the vehicle will attempt the fly-by pickup is received at block1614. A notification indicating to the passenger that the vehicle willattempt the fly-by pickup is displayed on the display based on thereceived confirmation at block 1616.

FIG. 17 is an example flow diagram 1700 in accordance with some of theaspects described above that may be performed by one or more computingdevices such as vehicle computing devices 110. For example, at block1702, dispatch instructions dispatching a driverless vehicle to apredetermined pickup area in order to pick up a passenger are received.The vehicle is maneuvered towards the predetermined pickup area to pickup the passenger at block 1704. While maneuvering the vehicle, alocation of a client computing device of the passenger is received atblock 1706. An indication that the passenger is interested in a fly-bypickup in real time is received at block 1706. The fly-by pickup allowsthe passenger to safely enter the vehicle at a location outside of thepredetermined pickup area and prior to vehicle reaching thepredetermined pickup area at block 1708. The fly-by pickup is determinedto be appropriate based on at least the location of the client computingdevice and the indication at block 1710. The vehicle is furthermaneuvered in order to attempt the fly-by pickup based on thedetermination at block 1712.

Unless otherwise stated, the foregoing alternative examples are notmutually exclusive, but may be implemented in various combinations toachieve unique advantages. As these and other variations andcombinations of the features discussed above can be utilized withoutdeparting from the subject matter defined by the claims, the foregoingdescription of the embodiments should be taken by way of illustrationrather than by way of limitation of the subject matter defined by theclaims. In addition, the provision of the examples described herein, aswell as clauses phrased as “such as,” “including” and the like, shouldnot be interpreted as limiting the subject matter of the claims to thespecific examples; rather, the examples are intended to illustrate onlyone of many possible embodiments. Further, the same reference numbers indifferent drawings can identify the same or similar elements.

The invention claimed is:
 1. A method for arranging a pickup between adriverless vehicle and a passenger, the method comprising: sending, byone or more processors of a client computing device associated with thepassenger, a request for the vehicle to pick up the passenger at apickup area; receiving, by the one or more processors, confirmation thatthe vehicle has been dispatched to the pickup area; receiving after thevehicle has been dispatched, by the one or more processors, locationinformation identifying a location of the vehicle as the vehicle ismoving towards the pickup area; displaying, by the one or moreprocessors on a display of the client computing device, the location ofthe vehicle and a location of the client computing device on a map,wherein displaying the location of the vehicle includes displaying afirst indicator on the map and displaying the location of the clientcomputing device includes displaying a second indicator on the map;receiving, by the one or more processors, updated location informationfor the vehicle; determining, by the one or more processors, updatedlocation information for the client computing device using a sensor ofthe client computing device; moving the first and second indicators onthe map based on the updated location information for the vehicle andthe updated location information for the client computing device; whenedges of the first and second indicators are moved to intersect oneanother, displaying a notification on the display indicating that thepassenger is able to attempt a fly-by pickup that allows the passengerto safely enter the vehicle at a location outside of the pickup area andprior to the vehicle having reached the pickup area; in response todisplaying the notification, receiving user input indicatingconfirmation to attempt the fly-by pickup; identifying, by the one ormore processors, an indication that the passenger is interested in thefly-by pickup in real time wherein the indication includes theconfirmation to attempt the fly-by pickup; sending, by the one or moreprocessors, the indication to one or more processors of a computingdevice associated with the vehicle; receiving, by the one or moreprocessors, confirmation that the vehicle will attempt the fly-bypickup; and displaying on the display, by the one or more processors, anotification indicating to the passenger that the vehicle will attemptthe fly-by pickup based on the received confirmation.
 2. The method ofclaim 1, wherein the method further comprising, receiving user inputindicating a tap at a location on the map, and wherein the indicationincludes a location on the map that corresponds to the user input, suchthat the location on the map is sent to the computing device associatedwith the vehicle.
 3. The method of claim 1, wherein the computing deviceis incorporated into the vehicle.
 4. The method of claim 1, furthercomprising, receiving, from a sensor of the client computing device,information indicating that the client computing device is currentlybeing moved, and wherein the indication includes the informationindicating that the client computing device is currently moving.
 5. Themethod of claim 1, further comprising receiving, from the computingdevice associated with the vehicle, a request for the passenger toperform a task in order to facilitate the fly-by pickup, wherein theindication includes confirmation that the task has been performed by thepassenger.
 6. The method of claim 5, wherein the task includes capturingan image of the passenger's surroundings and sending the image to thevehicle, and the method further comprises using the image to attempt thefly-by pickup.
 7. The method of claim 5, wherein the task includesproviding a verbal description of the passenger, and the method furthercomprises using the verbal description of the passenger to attempt thefly-by pickup.
 8. The method of claim 5, wherein the task includesplaying a sound through a speaker of the client computing device, andthe method further comprises using, by the one or more processors, thesound to maneuver the vehicle towards the passenger in order to attemptthe fly-by pickup.
 9. The method of claim 5, wherein the task includeswalking towards the vehicle, and the method further comprises stoppingthe vehicle, by the one or more processors, in order to allow thepassenger to enter the vehicle.
 10. The method of claim 1, wherein theindication includes information identifying that the passenger iswaiting by a curb outside of the pickup area.
 11. The method of claim 1,further comprising receiving, by the one or more processors, an image ofthe passenger from captured by a camera of the vehicle to indicate thatthe vehicle has identified the passenger as being nearby the vehicle.12. The method of claim 1, further comprising receiving a request forconfirmation that the passenger is interested in attempting the fly-bypickup, and wherein the indication is sent in response to the receivedrequest.
 13. The method of claim 1, wherein the first indicator isdisplayed as a first bubble and the second indicator is displayed as asecond bubble.
 14. The method of claim 13, wherein the first bubbleincludes a representation of the vehicle within the first bubble. 15.The method of claim 13, wherein the second bubble includes a markerwithin the first bubble, the marker representing the location of theclient computing device.
 16. The method of claim 1, further comprisingwhen the edges of the first and second indicators are moved to intersectone another, flashing the display at least once.
 17. The method of claim1, wherein the notification includes a request for the passenger toconfirm interesting in the fly-by pickup.
 18. The method of claim 1,wherein moving the first and second indicators on the map providesinformation about a relationship between the vehicle and the clientcomputing device.
 19. A system comprising a client computing devicehaving: a display; and one or more processors configured to: send arequest for a vehicle to pick up a passenger at a pickup area; receiveconfirmation that the vehicle has been dispatched to the pickup area;receive, after the vehicle has been dispatched, location informationidentifying a location of the vehicle as the vehicle is moving towardsthe pickup area; display on a display of the client computing device,the location of the vehicle and a location of the client computingdevice on a map, wherein displaying the location of the vehicle includesdisplaying a first indicator on the map and displaying the location ofthe client computing device includes displaying a second indicator onthe map; receive updated location information for the vehicle; determineupdated location information for the client computing device using asensor of the client computing device; move the first and secondindicators on the map based on the updated location information for thevehicle and the updated location information for the client computingdevice; when edges of the first and second indicators are moved tointersect one another, display a notification on the display indicatingthat the passenger is able to attempt a fly-by pickup that allows thepassenger to safely enter the vehicle at a location outside of thepickup area and prior to the vehicle having reached the pickup area; inresponse to displaying the notification, receive user input indicatingconfirmation to attempt the fly-by pickup; identify an indication thatthe passenger is interested in the fly-by pickup in real time whereinthe indication includes the confirmation to attempt the fly-by pickup;send the indication to one or more processors of a computing deviceassociated with the vehicle; receive confirmation that the vehicle willattempt the fly-by pickup; and display on the display a notificationindicating to the passenger that the vehicle will attempt the fly-bypickup based on the received confirmation.
 20. The system of claim 19,wherein the one or more processors are further configured to receive,from the computing device associated with the vehicle, a request for thepassenger to perform a task in order to facilitate the fly-by pickup,wherein the indication includes confirmation that the task has beenperformed by the passenger.